Journals
  Publication Years
  Keywords
Search within results Open Search
Please wait a minute...
For Selected: Toggle Thumbnails
Dynamic resource allocation strategy in Spark Streaming
LIU Bei, TAN Xinming, CAO Wenbin
Journal of Computer Applications    2017, 37 (6): 1574-1579.   DOI: 10.11772/j.issn.1001-9081.2017.06.1574
Abstract797)      PDF (982KB)(561)       Save
The existing resource allocation strategy has long resource adjustment cycle and cannot sufficiently meet the individual needs of different applications and users when Spark Streaming is selected as stream processing component in hybrid large-scale computing platform. In order to solve the problems, a Dynamic Resource Allocation strategy for Multi-application (DRAM) was proposed. The global variables were added to control the dynamic resource allocation process in DRAM. Firstly, the historical data feedback and the global variables were obtained. Then, whether increasing or decreasing the number of resources in each application was determined. Finally, the increase or decrease of resources was implemented. The experimental results show that, the proposed strategy can effectively adjust the resource quota, and reduce the processing delay compared with the original Spark platform strategies such as Streaming and Core under both cases of the stable data stream and the unstable data stream. The proposed strategy can also improve the utilization rate of the cluster resources.
Reference | Related Articles | Metrics
Design and implementation of configuration automatic system for computer core
LI Xinyou XU Tao LIU Bei
Journal of Computer Applications    2013, 33 (10): 2858-2860.  
Abstract525)      PDF (630KB)(605)       Save
The Computer Core Configuration (CCC) becomes one of the important means to improve the security of Internet computers, which has been brought under the spotlight in the field of information security. Based on the concept and the method of core configuration, an automatic way to implement CCC by Agent was described, a system composed of integrated functions such as configuration editing, verifying, dispatching and executing, and compliance monitoring was developed, which was suitable for organization to protect their computers against attack.
Related Articles | Metrics